AutoML Leaderboard

Best model	name	model_type	metric_type	metric_value	train_time
	1_Baseline	Baseline	auc	0.5	0.95
	2_DecisionTree	Decision Tree	auc	0.691806	7.97
	3_Default_LightGBM	LightGBM	auc	0.72954	6.21
	4_Default_Xgboost	Xgboost	auc	0.729286	30.41
	5_Default_CatBoost	CatBoost	auc	0.738735	3.82
	6_Default_NeuralNetwork	Neural Network	auc	0.738246	4.55
	7_Default_RandomForest	Random Forest	auc	0.725593	14.92
	8_Default_ExtraTrees	Extra Trees	auc	0.726322	10.85
the best	Ensemble	Ensemble	auc	0.741053	0.83

AutoML Performance

AutoML Performance Boxplot

Features Importance

Spearman Correlation of Models

Summary of 5_Default_CatBoost

<< Go back

CatBoost

• n_jobs: -1
• learning_rate: 0.1
• depth: 6
• rsm: 1
• loss_function: Logloss
• eval_metric: AUC
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

auc

Training time

3.0 seconds

Metric details

	score	threshold
logloss	0.600703	nan
auc	0.738735	nan
f1	0.705256	0.372437
accuracy	0.68028	0.524116
precision	0.898734	0.833308
recall	1	0.0525236
mcc	0.361119	0.524116

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.600703	nan
auc	0.738735	nan
f1	0.653165	0.524116
accuracy	0.68028	0.524116
precision	0.696356	0.524116
recall	0.615018	0.524116
mcc	0.361119	0.524116

Confusion matrix (at threshold=0.524116)

	Predicted as 0	Predicted as 1
Labeled as 0	1300	450
Labeled as 1	646	1032

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 8_Default_ExtraTrees

<< Go back

Extra Trees Classifier (Extra Trees)

• n_jobs: -1
• criterion: gini
• max_features: 0.9
• min_samples_split: 30
• max_depth: 4
• eval_metric_name: auc
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

auc

Training time

9.9 seconds

Metric details

	score	threshold
logloss	0.615822	nan
auc	0.726322	nan
f1	0.698202	0.380001
accuracy	0.670362	0.49277
precision	0.84	0.74073
recall	1	0.13811
mcc	0.340157	0.49277

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.615822	nan
auc	0.726322	nan
f1	0.655067	0.49277
accuracy	0.670362	0.49277
precision	0.671464	0.49277
recall	0.639452	0.49277
mcc	0.340157	0.49277

Confusion matrix (at threshold=0.49277)

	Predicted as 0	Predicted as 1
Labeled as 0	1225	525
Labeled as 1	605	1073

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Decision plots

<< Go back

Summary of 1_Baseline

<< Go back

Baseline Classifier (Baseline)

• n_jobs: -1
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

auc

Training time

0.3 seconds

Metric details

	score	threshold
logloss	0.692927	nan
auc	0.5	nan
f1	0.657266	0.440414
accuracy	0.489498	0.440414
precision	0.489498	0.440414
recall	1	0.440414
mcc	0	0.440414

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.692927	nan
auc	0.5	nan
f1	0.657266	0.440414
accuracy	0.489498	0.440414
precision	0.489498	0.440414
recall	1	0.440414
mcc	0	0.440414

Confusion matrix (at threshold=0.440414)

	Predicted as 0	Predicted as 1
Labeled as 0	0	1750
Labeled as 1	0	1678

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of Ensemble

<< Go back

Ensemble structure

Model	Weight
5_Default_CatBoost	1
6_Default_NeuralNetwork	1

Metric details

	score	threshold
logloss	0.601162	nan
auc	0.741053	nan
f1	0.705693	0.344842
accuracy	0.677655	0.563138
precision	0.881818	0.863075
recall	1	0.0684266
mcc	0.356771	0.563138

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.601162	nan
auc	0.741053	nan
f1	0.644809	0.563138
accuracy	0.677655	0.563138
precision	0.69993	0.563138
recall	0.597735	0.563138
mcc	0.356771	0.563138

Confusion matrix (at threshold=0.563138)

	Predicted as 0	Predicted as 1
Labeled as 0	1320	430
Labeled as 1	675	1003

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 2_DecisionTree

<< Go back

Decision Tree

• n_jobs: -1
• criterion: gini
• max_depth: 3
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

auc

Training time

7.0 seconds

Metric details

	score	threshold
logloss	0.630016	nan
auc	0.691806	nan
f1	0.657266	0.152776
accuracy	0.489498	0.152776
precision	0.489498	0.152776
recall	1	0.152776
mcc	0	0.152776

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.630016	nan
auc	0.691806	nan
f1	0.657266	0.152776
accuracy	0.489498	0.152776
precision	0.489498	0.152776
recall	1	0.152776
mcc	0	0.152776

Confusion matrix (at threshold=0.152776)

	Predicted as 0	Predicted as 1
Labeled as 0	0	1750
Labeled as 1	0	1678

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

<< Go back

Summary of 3_Default_LightGBM

<< Go back

LightGBM

• n_jobs: -1
• objective: binary
• num_leaves: 63
• learning_rate: 0.05
• feature_fraction: 0.9
• bagging_fraction: 0.9
• min_data_in_leaf: 10
• metric: auc
• custom_eval_metric_name: None
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

auc

Training time

5.3 seconds

Metric details

	score	threshold
logloss	0.607881	nan
auc	0.72954	nan
f1	0.697651	0.395582
accuracy	0.674154	0.489103
precision	0.841584	0.804517
recall	1	0.0624953
mcc	0.348474	0.487772

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.607881	nan
auc	0.72954	nan
f1	0.67021	0.489103
accuracy	0.674154	0.489103
precision	0.664131	0.489103
recall	0.6764	0.489103
mcc	0.348325	0.489103

Confusion matrix (at threshold=0.489103)

	Predicted as 0	Predicted as 1
Labeled as 0	1176	574
Labeled as 1	543	1135

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Decision plots

<< Go back

Summary of 6_Default_NeuralNetwork

<< Go back

Neural Network

• n_jobs: -1
• dense_1_size: 32
• dense_2_size: 16
• learning_rate: 0.05
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

auc

Training time

3.7 seconds

Metric details

	score	threshold
logloss	0.61231	nan
auc	0.738246	nan
f1	0.703954	0.395631
accuracy	0.675029	0.510942
precision	0.849741	0.892842
recall	1	0.0758362
mcc	0.354338	0.440811

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.61231	nan
auc	0.738246	nan
f1	0.676351	0.510942
accuracy	0.675029	0.510942
precision	0.659864	0.510942
recall	0.693683	0.510942
mcc	0.350898	0.510942

Confusion matrix (at threshold=0.510942)

	Predicted as 0	Predicted as 1
Labeled as 0	1150	600
Labeled as 1	514	1164

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 7_Default_RandomForest

<< Go back

Random Forest

• n_jobs: -1
• criterion: gini
• max_features: 0.9
• min_samples_split: 30
• max_depth: 4
• eval_metric_name: auc
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

auc

Training time

14.0 seconds

Metric details

	score	threshold
logloss	0.612224	nan
auc	0.725593	nan
f1	0.700493	0.35222
accuracy	0.669195	0.458205
precision	0.823529	0.784412
recall	1	0.0992961
mcc	0.341151	0.458205

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.612224	nan
auc	0.725593	nan
f1	0.678389	0.458205
accuracy	0.669195	0.458205
precision	0.647186	0.458205
recall	0.712753	0.458205
mcc	0.341151	0.458205

Confusion matrix (at threshold=0.458205)

	Predicted as 0	Predicted as 1
Labeled as 0	1098	652
Labeled as 1	482	1196

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Decision plots

<< Go back

Summary of 4_Default_Xgboost

<< Go back

Extreme Gradient Boosting (Xgboost)

• n_jobs: -1
• objective: binary:logistic
• eta: 0.075
• max_depth: 6
• min_child_weight: 1
• subsample: 1.0
• colsample_bytree: 1.0
• eval_metric: auc
• explain_level: 2

Validation

• validation_type: split
• train_ratio: 0.75
• shuffle: True
• stratify: True

Optimized metric

auc

Training time

29.4 seconds

Metric details

	score	threshold
logloss	0.60775	nan
auc	0.729286	nan
f1	0.698967	0.376195
accuracy	0.672404	0.488988
precision	0.835443	0.837847
recall	1	0.047195
mcc	0.344713	0.488988

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.60775	nan
auc	0.729286	nan
f1	0.667456	0.488988
accuracy	0.672404	0.488988
precision	0.663331	0.488988
recall	0.671633	0.488988
mcc	0.344713	0.488988

Confusion matrix (at threshold=0.488988)

	Predicted as 0	Predicted as 1
Labeled as 0	1178	572
Labeled as 1	551	1127

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Decision plots

<< Go back